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The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply w^ithin the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)K Responsive to communication(s) filed on 03 November 2003 . 
2a)n This action is FINAL. 2b)E] This action is non-final. 

3) n Since this application is in condition for allowance except for fomnal matters, prosecution as to the nnerits is 

closed in accordance with the practice under Expa/te Quay/e, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) K Claim(s) 1-12 and 21-28 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) n Claim(s) is/are allowed. 

6) 13 Claim(s) 1-3,5-7.10 and 12 is/are rejected. 

7) 13 Claim(s) 4. 8. 9. 11, and 21-28 is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) n The specification is objected to by the Examiner. 

10) ^ The drawing(s) filed on 08 September 2000 is/are: a)n accepted or b)[3 objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

11) n The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 
Priority under 35 U.S.C. §§119 and 120 

12) 0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)nAII b)n Some*c)n None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

1 3) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 1 9(e) (to a provisional application) 

since a specific reference was included in the first sentence of the specification or in an Application Data Sheet. 
37 CFR 1.78. 

a) □ The translation of the foreign language provisional application has been received. 

14) n Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 since a specific 

reference was included in the first sentence of the specification or in an Application Data Sheet. 37 CFR 1 .78. 



Attachment(s) 

1) ^ Notice of References Cited (PTO-892) 

2) n Notice of Draflsperson's Patent Drawing Review (PTO-948) 

3) ^ Information Disclosure Statement(s) (PTO-1449) Paper No(s) 2 . 



4) □ Interview Summary (PTO-413) Paper No(s). 

5) n Notice of Informal Patent Application (PTO-152) 

6) □ Other: 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 11-03) 
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DETAILED ACTION 



1. 



The response received on November 3, 2003 has been placed in the file and was 



considered by the examiner. An action on the merits follows. 



Election/Restrictions 



2. 



Applicant's election without traverse of the invention of Group I in Paper No. 5 is 



acknowledged. 



Information Disclosure Statement 



3. 



The examiner requests copies of several references mentioned in the specification. These 



references are pertinent to the examination of this application. Full consideration of this prior art 
is essential. The references are Adelson et al. (the article titled "Ordinal characteristics of 
transparency") cited on_page 9, lines 25-26 of the specification, Blinn (the article titled 
"Compositing, part 1: rneory) cited on page 10, lines 4-6 of the specification, and Debevec et al. 
(the article titled "Recovering high dynamic range radiance maps fi*om photographs") cited on 
page 11, lines 23-24 or the specification. 



4. The drawings are objected to because the description of Figs. 3D and 3E on page 14, 
lines 18-20 calls for red curves. Which curves are the red curves? The examiner is unable to 
determine this feature since the figures are black and white. A proposed drawing correction, 
corrected drawings, or clarification of this issue are required in reply to the Office action to avoid 
abandonment of the application. The objection to the drawings will not be held in abeyance. 

Claim Objections 

5. The following quotation of 37 CFR § 1.75(a) is the basis of objection: 



Drawings 
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(a) The specification must conclude with a claim particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention or discovery. 



6. Claims 1-12 and 21-28 are objected to under 37 CFR § 1.75 as failing to particularly 
point out and distinctly claim the subject matter which the applicant regards as his invention or 



Claim 1 recites the limitation "the layers" in line 10. There is insufficient antecedent 
basis for this limitation in the claim. Is the limitation "the layers" referring to the "primary and 
secondary layers"? Is the limitation "the layers" referring to different, newly estimated layers? 
Clarification of this issue is required. For examination purposes, the examiner has interpreted 
this limitation as reading "the primary and secondary layers". 

Claim 1 1 recites the limitation "the unconstrained least-square solution" in lines 1-2. 
There is insufficient antecedent basis for this limitation in the claim. 

Claims 2-10, 12, and 21-28 are objected to as depending from objected to claims. 



7. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

8. Claims 1-3, 6, 7, and 12 are rejected under 35 U.S.C. 102(b) as being anticipated by 
Wang et al. (U.S. Patent No. 5,557,684 A). 

As applied to claim 1, Wang et al. discloses a method for performing layer extraction 
from multiple images containing reflections and transparencies, comprising: computing a 
primary motion estimate (see column 5, lines 32-41 : The reference describes that a motion 



discovery. 



Claim Rejections - 35 USC §102 
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estimator determines regions of coherent motion in the image and forms a set of associated 
motion models for the regions. The first of these motion models would be the primary motion 
estimate.); estimating a primary layer associated with the primary motion estimate (see column 
6, lines 9-1 1 and column 4, lines 43-48: The reference describes a layer extraction processor that 
produces one layer based on the region information and the motion model. The first motion 
model (i.e. primary motion estimate) would be used to extract the first or background layer (i.e. 
primary layer).); computing a secondary motion estimate (see column 5, lines 32-41 : The 
reference describes that a motion estimator determines regions of coherent motion in the image 
and forms a set of associated motion models for the regions. The second motion model would be 
the secondary motion estimate.); estimating a secondary layer associated with the secondary 
motion estimate (see column 6, lines 9-11 and column 4, lines 43-48: The reference describes a 
layer extraction processor that produces one layer based on the region information and the 
motion model. The second motion model (i.e. secondary motion estimate) would be used to 
extract the second layer (i.e. secondary layer).); and iteratively refining lower and upper bounds 
on the primary and secondary layers to estimate the layers (see column 1 1, line 67 - column 12, 
line 6: The reference describes that the intensity values (i.e. lower and upper bounds) of the 
layers are combined overtime (i.e. iteratively refined) to form (i.e. estimate) the final layers.). 

As applied to claim 2, Wang et al. discloses improving the motion estimates using motion 
re-estimation (see column 6, lines 15-19: The reference describes that the processor combines 
motion parameters in one fi*ame with motion parameters in the next fi"ame to produce a modified 
(i.e. improved) set of motion parameters.). 
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As applied to claim 3, Wang et al. discloses stabilizing the images with respect to the 
primary layer (see column 4, lines 43-48: The reference describes that a given image can be 
segmented into four layers. The essentially motionless background layer (i.e. primary layer) 
stabilizes the images.). 

As applied to claim 6, Wang et al discloses that iteratively refining includes recovering 
the primary layer and the secondary layer of the images (see Fig. 1 : The reference shoves that a 
primary 4a layer and secondary la layer are recovered.). 

As applied to claim 7, Wang et al. discloses that the multiple images form a video 
sequence containing reflections and transparencies (see column 4, lines 27-30: The reference 
describes the use of frames from a video sequence. These frames include reflections and 
transparencies.). 

As applied to claim 12, which merely calls for a computer-readable medium having 
computer executable instructions for performing the method of claim 1, Wang et al. discloses 
such a computer-readable medium since all of the processing in Wang et al. is performed by 
computer (see Fig. 3). 

Claim Rejections - 35 USC §103 

9. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

10. Claims 5 and 10 are rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Wang et al. (U.S. Patent No. 5,557,684 A) and Kaup et al. (the article titled "A 
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Stochastic Model for Image Segmentation Involving Constrained Least Squares Estimation"). 
The arguments as to the relevance of Wang et al in the rejection of claim 1 above are 
incorporated herein. 

Claim 5 calls for the use of constrained least squares to optimally recover the layer 
images. While Wang et al. describes the use of least squares estimation to estimate the motion 
(see column 7, lines 13-15), the reference does not describe a constrained least squares process 
for optimally recovering the layer images. However, Kaup et al. disclose such a process (see 
column 2, section 3 : The reference describes the use of a constrained least-squares estimation for 
optimizing image segmentation.). 

As applied to claim 10, Wang et al. discloses alternating the least-squares optimization of 
layer values with motion re-estimation (see column 6, lines 15-19: The reference describes that 
the processor combines motion parameters in one frame with motion parameters in the next 
frame to produce a modified set of motion parameters. This modified set of motion parameters is 
equivalent to a motion re-estimation.). 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to modify Wang et al. by adding the use of constrained least squares as 
taught in Kaup et al. because the use of such a calculation because this calculation produces 
"segmentation results that are visually convincing and well suited for image analysis" (see Kaup 
et al.: column 2, section 3). 
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Allowable Subject Matter 



11. Claims 4, 8, 9, 1 1, and 21-28 would be allowable if rewritten to overcome the 
objection(s) under 37 CFR § 1.75(aX set forth in this Office action and to include all of the 
limitations of the base claim and any intervening claims. 



12. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

Anandan et al. (U.S. Patent No. 6,049,619 A) is pertinent in that the reference discloses a 
method of detecting motion in an image that uses layers. 

Hanna et al. (U.S. Patent No. 6,269,175 Bl) is pertinent in that the reference discloses a 
method of enhancing regions in an image that uses layers. 

Tao et al. (the article titled "Dynamic Layer Representation with Applications to 
Tracking") is pertinent in that the reference discloses a method of using dynamic layers in an 
image for tracking a moving object. 

Torres et al. (the article titled "A Robust Motion Estimation and Segmentation Approach 
to Represent Moving Images with Layers") is pertinent in that the reference discloses the use of 
motion estimation and layers in a segmentation process. 

Szeliski et al. (the article titled "Stereo Matching with Transparency and Matting") is 
pertinent in that the reference discloses recovering different characteristics of visible surface 
elements. 



Conclusion 
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13. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ryan J. Miller whose telephone number is (703) 306-4142. The 
examiner can normally be reached on M-F 8:00-4:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Leo H. Boudreau can be reached on (703) 305-4706. The fax phone number for the 
organization where this application or proceeding is assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 305-4750. 




Ryan J. Miller 
Examiner 
Art Unit 2621 




Ryan J. Miller 




